To describe phenomena that occur at different time scales, computational models of the brain must incorporate different levels of abstraction. At time scales of approximately 1/3 of a second, orienting movements of the body play a crucial role in cognition and form a useful computational level embodiment level,” the constraints of the physical system determine the nature of cognitive operations. The key synergy is that at time scales of about 1/3 of a second, the natural sequentiality of body movements can (...) be matched to the natural computational economies of sequential decision systems through a system of implicit reference called deictic in which pointing movements are used to bind objects in the world to cognitive programs. This target article focuses on how deictic bindings make it possible to perform natural tasks. Deictic computation provides a mechanism for representing the essential features that link external sensory data with internal cognitive programs and motor actions. One of the central features of cognition, working memory, can be related to moment-by-moment dispositions of body features such as eye movements and hand movements. (shrink)

Animate vision systems have gaze control mechanisms that can actively position the camera coordinate system in response to physical stimuli. Compared to passive systems, animate systems show that visual computation can be vastly less expensive when considered in the larger context of behavior. The most important visual behavior is the ability to control the direction of gaze. This allows the use of very low resolution imaging that has a high virtual resolution. Using such a system in a controlled way provides (...) additional constraints that dramatically simplify the computations of early vision. Another important behavior is the way the environment “behaves‘. Animate systems under real-time constraints can further reduce their computational burden by using environmental cues that are perspicuous in the local context. A third source of economy is introduced when behaviors are learned. Because errors are rarely fatal, systems using learning algorithms can amortize computational cost over extended periods. Further economies can be achieved when the learning system uses indexical reference, which is a form of dynamic variable binding. Animate vision is a natural way of implementing this dynamic binding. (shrink)

The cerebral cortex is a rich and diverse structure that is the basis of intelligent behavior. One of the deepest mysteries of the function of cortex is that neural processing times are only about one hundred times as fast as the fastest response times for complex behavior. At the very least, this would seem to indicate that the cortex does massive amounts of parallel computation.This paper explores the hypothesis that an important part of the cortex can be modeled as a (...) connectionist computer that is especially suited for parallel problem solving. The connectionist computer uses a special representation, termed value unit encoding, that represents small subsets of parameters in a way that allows parallel access to many different parameter values. This computer can be thought of as computing hierarchies of sensorimotor invariants. The neural substrate can be interpreted as a commitment to data structures and algorithms that compute invariants fast enough to explain the behavioral response times. A detailed consideration of this model has several implications for the underlying anatomy and physiology. (shrink)

Ballard and Zhang offer a fascinating review of how computational models of human vision have evolved since David Marr proposed his Tri‐Level Hypothesis, with a focus on the refinement of algorithm descriptions over time.

Our seamless perception of the world depends very much on the slow time scales used by conscious perception. Time scales longer than one second are needed to assemble conscious experience. At time scales shorter than one second, this seamlessness quickly deteriorates. Numerous experiments reveal the fragmentary nature of the visual information used to construct visual experience. Models of how the brain manages these fragments use the construct of a routine, which is a task-specific fragment of a sensory-motor program. This paper (...) provides an overview of some of the experiments that test these models. Its aim is to show how the structures that they elucidate constrain the understanding of conscious perception. (shrink)

The majority of commentators agree that the time to focus on embodiment has arrived and that the disembodied approach that was taken from the birth of artificial intelligence is unlikely to provide a satisfactory account of the special features of human intelligence. In our Response, we begin by addressing the general comments and criticisms directed at the emerging enterprise of deictic and embodied cognition. In subsequent sections we examine the topics that constitute the core of the commentaries: embodiment mechanisms, dorsal (...) and ventral visual processing, eye movements, and learning. (shrink)

The cerebral cortex is a rich and diverse structure that is the basis of intelligent behavior. One of the deepest mysteries of the function of cortex is that neural processing times are only about one hundred times as fast as the fastest response times for complex behavior. At the very least, this would seem to indicate that the cortex does massive amounts of parallel computation.This paper explores the hypothesis that an important part of the cortex can be modeled as a (...) connectionist computer that is especially suited for parallel problem solving. The connectionist computer uses a special representation, termed value unit encoding, that represents small subsets of parameters in a way that allows parallel access to many different parameter values. This computer can be thought of as computing hierarchies of sensorimotor invariants. The neural substrate can be interpreted as a commitment to data structures and algorithms that compute invariants fast enough to explain the behavioral response times. A detailed consideration of this model has several implications for the underlying anatomy and physiology. (shrink)

Our computational studies of infant language learning estimate the inherent difficulty of Arbib's proposal. We show that body language provides a strikingly helpful scaffold for learning language that may be necessary but not sufficient, given the absence of sophisticated language in other species. The extraordinary language abilities of Homo sapiens must have evolved from other pressures, such as sexual selection.